Skip to content

feat(models): Ollama capabilities catalog + chat Thinking chip - #1152

Open
Sdaswani wants to merge 7 commits into
thunderbird:mainfrom
Sdaswani:feat/ollama-capabilities-thinking-chip
Open

feat(models): Ollama capabilities catalog + chat Thinking chip#1152
Sdaswani wants to merge 7 commits into
thunderbird:mainfrom
Sdaswani:feat/ollama-capabilities-thinking-chip

Conversation

@Sdaswani

Copy link
Copy Markdown
Contributor

Summary

  • When adding a Custom provider whose URL looks like Ollama (port 11434 or hostname contains ollama), prefer native GET /api/tags so tools / thinking / vision / context window prefills are richer than sparse /v1/models.
  • Soft-fail the probe (network error, non-Ollama JSON, empty catalog) and fall back to /v1/models so non-Ollama custom endpoints keep working.
  • Persist advertised capabilities into the model row (toolUsage, startWithReasoning, contextWindow) on create and edit, show them on model detail, and add a per-conversation Thinking chip in the composer for models with startWithReasoning === 1.
  • Chip off does not mutate the stored model (so the chip stays visible); it overrides the send path (startWithReasoning view + reasoningEffort: 'none' for OpenAI-compat / Ollama, and Pi thinkingLevel: 'off' in the built-in adapter).

Durable contract is the model row + session override — not “we are permanently bound to Ollama.” Autodetection is opportunistic.

Heuristic / failover notes

  • Probe gate: isLikelyOllamaBaseUrl — intentional incomplete match so llama.cpp / corporate OpenAI-compat URLs are not hit with an extra failed /api/tags.
  • Ollama on an uncommon port without ollama in the host still works via /v1/models (no rich caps).
  • /api/tags body is zod-validated (ollamaTagsResponseSchema envelope + per-tag ollamaTagModelSchema); malformed tags are skipped; empty/invalid → soft-fail.

Test plan

  • Unit: ollama-catalog origin/heuristic/mapping/schema
  • Unit: fetchModelsForProvider prefers /api/tags; fails over on throw and non-Ollama body
  • Unit: add-model form persists Ollama-advertised flags
  • Unit: thinking-session helpers + openaiCompatThinkingProviderOptions
  • Unit: withThinkingSessionOverride (ACP / Pi path)
  • Unit: ThinkingChip a11y pressed/toggle
  • Manual: Custom URL http://localhost:11434/v1 → catalog shows tools/thinking/context; create model; chip appears in chat
  • Manual: toggle Thinking off → send does not stream reasoning (Ollama thinking model)
  • Manual: Custom URL non-Ollama (e.g. llama.cpp other port) → no /api/tags probe failure in UX; /v1/models path works
  • Manual: Edit model → change Thinking / Tools / Context window → detail + chip reflect updates
  • Manual: switch to a non-thinking model → chip hidden; prior chat’s thinkingEnabled does not leak (reset on model switch)

Reviewer notes

Addressed prior review:

  1. Dropped ALL_CAPS REASONING_OFF (inlined 0)
  2. Removed dead contextLengthFromModelInfo; wired ollamaTagsResponseSchema into the fetch path (envelope stays unknown[] for per-tag skip)
  3. Removed as never casts on stubJsonResponse mocks
  4. Spread ResolvedPiModel consistently in both withThinkingSessionOverride branches

Made with Cursor

Sdaswani and others added 4 commits July 28, 2026 19:28
Co-authored-by: Cursor <cursoragent@cursor.com>
Zod-validate /api/tags at the network boundary, replace the startWithReasoning
cast with withThinkingDisabledForSend, drop redundant TooltipProvider nesting,
and document the Ollama URL probe heuristic trade-off.

Co-authored-by: Cursor <cursoragent@cursor.com>
Cover /api/tags soft-fail → /v1/models, expose Thinking/tools/context on
edit, and lock Thinking chip + reasoningEffort provider options in unit tests.

Co-authored-by: Cursor <cursoragent@cursor.com>
…se, DRY schema

Inline reasoning-off flag, wire ollamaTagsResponseSchema into the fetch path,
remove unused model_info helper, drop as-never test casts, and spread
ResolvedPiModel consistently in withThinkingSessionOverride.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@github-actions

github-actions Bot commented Jul 29, 2026

Copy link
Copy Markdown

Preview environment deployed 🚀

Service URL
Marketing / blog / docs https://thunderbolt-pr-1152.preview.thunderbolt.io
App https://app-pr-1152.preview.thunderbolt.io
API https://api-pr-1152.preview.thunderbolt.io
Keycloak https://auth-pr-1152.preview.thunderbolt.io
PowerSync https://powersync-pr-1152.preview.thunderbolt.io

Stack: preview-pr-1152 · Commit: f9e5ee6f1d451babe02e81a9f3bdda2b8dccfcb7

Auto-destroys on PR close/merge. Login via the bundled Keycloak realm — demo@thunderbolt.io / demo by default.

- model.startWithReasoning means show/enable thinking; middleware
  startWithReasoning means treat content as reasoning until </think>
- Ollama thinking models emit a native reasoning field and plain answer
  content with no closing tag, so wiring the chip flag through left the
  spinner stuck with the reply in the muted Thinking pane
- show a Thinking label while streaming with no active tool

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant